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of  networks,  each  one  being  a  strict  augmentation  of  the  previous  one,  which 
control  a  six  legged  walking  machine  capable  of  walking  over  rough  terrain 
and  following  a  person  passively  sensed  in  the  infrared  spectrum.  As  the  com¬ 
pletely  decentralized  networks  are  augmented,  the  robot's  performance  and 
behavior  repetoire  demonstrably  improve.  The  rationale  for  such  demonstra¬ 
tions  is  that  they  may  provide  a  hint  as  to  the  requirements  for  automatically 
building  massive  networks  to  carry  out  complex  sensory-motor  tasks.  The  ex¬ 
periments  with  an  actual  robot  ensure  that  an  essence  of  reality  is  maintained 
and  that  no  critical  disabling  problems  have  been  ignored. 
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Introduction 

In  earlier  work,  [l],  [2],  we  have  demonstrated  complex  control  systems  for 
mobile  robots  built  from  completely  distributed  networks  of  augmented  finite 
state  machines.  In  this  paper  we  demonstrate  that  these  techniques  can  be 
used  to  incrementally  build  complex  systems  integrating  relatively  large  num¬ 
bers  of  sensory  inputs  and  large  numbers  of  actuator  outputs.  Each  step  in 
the  construction  is  purely  incremental,  but  nevertheless  along  the  wav  viable 
control  svstems  are  left  at  each  step,  before  the  next  little  piece  of  network  is 
added.  Additionally  we  demonstrate  how  complex  behaviors,  such  as  walking, 
can  emerge  from  a  network  of  rat  her  simple  relieves  wit  h  ljn  |e  cent  rai  cont  ml. 
This  contradicts  vague  hypotheses  made  to  the  contrary  during  the  studv  of 
insect  walking  (e.g.  [3].  page  1  12). 


The  Subsumption  Architecture 

l  lie  subsumption  architect  ure[l]  prov  ides  .m  incremental  method  for  build¬ 
ing  robot  control  systems  linking  perception  to  action.  A  properly  designed 
network  of  finite  state  machines,  augmented  with  internal  timers,  provides  a 
robot  with  a  certain  level  of  performance,  and  a  repet  oire  of  behaviors.  The 
architecture  provides  mechanisms  to  augment  such  networks  in  a  purely  incre¬ 
mental  way  to  improve  the  robot's  performance  on  tasks  and  to  increase  the 
range  of  tasks  it  can  perform.  At  an  architectural  level,  the  robot's  control 
system  is  expressed  as  a  series  of  layers,  each  specifying  a  behavior  pattern  for 
the  robot,  and  each  implemented  as  a  network  of  message  passing  augmented 
finite  state  machines.  The  network  can  be  thought  of  as  an  explicit  wiring 
diagram  connected  outputs  of  some  machines  to  inputs  of  others  with  wires 
that  can  transmit  messages.  In  the  implementation  of  the  architecture  on  the 
walking  robot  the  messages  are  limited  to  *  hits. 

Each  augmented  finite  state  machine  (AFSM).  figure  1.  has  a  set  of  reg¬ 
isters  and  a  set  of  timers,  or  alarm  clocks,  connected  to  a  conventional  finite 
state  machine  which  can  control  a  combinatorial  network  fed  by  the  regis¬ 
ters.  Registers  can  be  written  by  attaching  input  wires  to  them  and  sending 
messages  from  other  machines.  The  messages  get  written  into  them  replacing 
anv  existing  contents.  The  arrival  of  a  message,  or  the  expiration  of  a  timer, 
can  trigger  a  change  of  state  in  the  interior  finite  state  machine.  Finite  state 
machine  states  can  eit her  wait  on  some  event .  conditionally  dispatch  to  one  of 
two  other  states  based  on  some  combinational  predicate  on  the  registers,  or 
compute  a  combinatorial  function  of  the  registers  directing  the  result  either 
back  to  one  of  the  regisi  r  r  *  ►  r  to  an  <ut  put  •  >  t  t  fie  a  ir_i  me  n  i  efi  finite  -late  ma 


Augmented  FSM 


Figure  1.  An  augmented  finite  state  machine  consists  of  registers,  alarm  clocks, 
a  combinatorial  network  and  a  regular  finite  state  machine.  Input  messages  are 
delivered  to  registers,  and  messages  can  be  generated  on  output  wires.  AFSMs  are 
wired  together  in  networks  of  message  passing  wires.  As  new  wires  are  added  to  a 
network,  they  can  be  connected  to  existing  registers,  they  can  inhibit  outputs  and 
they  can  suppress  inputs. 


chine.  Some  AFSMs  connect  direct  I v  to  robot  hardware.  Sensors  deposit  their 
v  aloes  to  cert  a  in  regi-t  rr- .  .uni  i  <  1 1  a  in  outputs  direct  commands  to  ac  t  uat  ofs. 


4 


A  series  of  layers  of  such  machines  can  be  augmented  by  adding  new 
machines  and  connecting  them  into  the  existing  network  in  the  ways  shown 
in  figure  1.  New  inputs  can  be  connected  to  existing  registers,  which  might 
previously  have  contained  a  constant.  New  machines  can  inhibit  existing  out¬ 
puts  or  suppress  existing  inputs,  by  being  attached  as  side-taps  to  existing 
wires  (figure  1,  circled  "i’).  When  a  message  arrives  on  an  inhibitory  side- tap 
no  messages  can  travel  along  the  existing  wire  for  some  short  time  period. 
To  maintain  inhibition  there  must  be  a  continuous  flow  of  messages  along  the 
new  wire.  (In  previous  versions  of  the  subsumption  architecture^]  explicit, 
long,  time  periods  had  to  be  specified  for  inhibition  or  suppression  with  single 
shot  messages.  Recent  work  has  suggested  this  better  approach  [4].)  When 
a  message  arrives  on  a  suppressing  side-tap  (figure  1,  circled  's’),  again  no 
messages  are  allowed  to  flow  from  the  original  source  for  some  small  time 
period,  but  now  the  suppressing  message  is  gated  through  and  it  masquer¬ 
ades  as  having  come  from  the  original  source.  Again,  a  continuous  supply  of 
suppressing  messages  is  required  to  maintain  control  of  a  side-tapped  wire. 
One  last  mechanism  for  merging  two  wires  is  called  defaulting  (indicated  in 
wiring  diagrams  by  a  circled  hi’).  This  is  just  like  the  suppression  case,  except 
that  the  original  wire,  rather  than  the  new  side-tapping  wire,  is  able  to  wrest 
control  of  messages  sent  to  the  destination. 

All  clocks  in  a  subsumption  system  have  approximately  the  same  tick 
period  (0.04  seconds  on  the  walking  robot),  but  neither  they  nor  messages  are 
synchronous.  The  fastest  possible  rate  of  sending  messages  along  a  wire  is  one 
per  clock  tick.  The  time  periods  used  for  both  inibition  and  suppression  are 
two  clock  ticks.  Thus,  a  side-tapping  wire  with  messages  being  sent  at  the 
maximum  rate  can  maintain  control  of  its  host  wire. 


The  Networks  and  Emergent  Behaviors 

The  six  legged  robot  is  shown  in  figure  2.  We  refer  to  the  motors  on  each  leg 
as  an  a  motor  (for  advance)  which  swings  the  leg  back  and  forth,  and  a  ji 
motor  (for  balance)  which  lifts  the  leg  up  and  down. 

Figure  3  shows  a  network  of  57  augmented  finite  state  machines  which  was 
built  incrementally  and  can  be  run  incrementally  by  selectively  deactivating 
later  AFSMs.  The  AFSMs  without  bands  on  top  are  repeated  six  times, 
once  for  each  leg.  The  ASFMs  with  solid  bands  are  unique  and  comprise 
the  only  centrai  control  in  making  the  robot  walk,  steer  and  follow  targets. 
The  ASFMs  with  striped  bands  are  duplicated  twice  each  and  are  specific  to 
particular  legs. 


Figure  2.  The  six  legged  robot  is  about  .Too rn  long,  has  a  leg  span  of  2 5c in,  and 
weighs  approximately  1kg.  Each  log  is  rigid  and  is  attached  at  a  shoulder  joint 
with  two  degrees  of  rotational  freedom,  driven  bv  two  orthoganallv  mounted  model 
airplane  position  controllable  servo  motors.  An  error  signal  has  been  tapped  from 
the  internal  servo  circuitry  to  provide  crude  force  measurement  (5  bits,  including 
sign)  on  each  axis,  when  the  leg  is  not  in  motion  around  that  axis.  Other  sensors 
are  two  front  whiskers,  two  four  bit  indinomters  (pitch  and  roll),  and  six  forward 
looking  passive  pyroelectric  infrared  sensors.  The  sensors  have  approximately  ti 
degrees  angular  resolution  and  are  arranged  over  a  15  degree  span.  There  are  four 
onboard  8  bit  microprocessors  linked  bv  a  d2.5Kbattd  token  ring.  The  total  memory 
usage  of  the  robot  is  about  lKbytes  of  RAM  and  lOKbytes  of  EPROM.  Three 
silver-zinc  batteries  fit  between  the  legs  to  make  the  robot  totally  self  contained. 

The  complete  network  can  be  built  incrementally  by  adding  AFSMs  to  an 
eidsting  network  producing  a  number  of  viable  robot  control  systems  itemized 
below.  All  additions  are  strictly  additive  with  no  need  to  change  any  existing 
structure.  Figure  4  shows  a  partially  constructed  version  of  the  network. 

/  Standup.  The  simplest  level  of  competence  for  the  robot  is  achieved 
with  just  two  AFSMs  per  leg,  alpha  pos  and  lx  ta  po These  two  machines 
use  a  register  to  hold  a  set  position  for  the  a  and  T  motors  respectively 
and  ensure  that  the  motors  are  sent  those  positions.  The  initial  values 
for  the  registers  are  such  that  on  power  up  the  robot  assumes  a  stance 
position.  The  AFSMs  also  provide  an  output  that  reports  t he  most  recent 
commanded  position  for  their  motor. 

2  Simple  walk.  A  number  of  simple  increments  to  ilii-  network  iv-uh  in 


Figure  3.  The  final  network  consists  of  57  augmented  finite  -ante  machines.  The 
AFSMs  without  bands  on  top  are  repeated  six  times,  once  for  eacli  leg.  The  ASF  Ms 
with  solid  bands  are  unique  and  comprise  the  only  central  control  in  making  the 
robot  walk,  steer  and  follow  targets.  The  AFSMs  with  striped  bands  are  duplicated 
twice  each  and  are  specific  to  particular  legs.  The  ASFMs  with  a  tilled  triangle  in 
their  bottom  right  corner  control  actuators.  Those  with  a  filled  triangle  in  their 
upper  left  corner  receive  inputs  from  sensors. 


one  which  lets  the  robot  walk.  First,  a  l< </  down  machine  for  each  leg 
is  added  which  notices  whenever  the  hg  is  not  in  the  down  position  and 


Figure  4.  A  strict  subset  of  the  full  network  enables  t  lie  robot  to  walk  without  any 
feedback.  It  pitches  and  rolls  significantly  as  it  walks  over  rough  terrain.  This  version 
of  the  network  contains  32  AFSMs.  30  of  these  comprise  six  identical  copies,  one 
for  each  leg,  of  a  network  of  five  AFSMs  which  are  purely  local  in  their  interactions 
with  a  leg.  The  last  two  machines  provide  all  the  global  coordination  necessary  to 
make  the  machine  walk:  one  tries  to  drive  the  sum  of  leg  swing  angles  (a  angles)  to 
zero,  and  the  other  sequences  lifting  of  individual  less 

writes  the  appropriate  ht  In  pa*  register  in  order  to  set  the  leg  down.  Then. 

a  single  alpha  hulnnct  machine  i-  added  which  monitors  t  lie  n  position. 


or  forward  swing  of  all  six  legs,  treating  straight  out  as  zero,  forward  as 
positive  and  backward  as  negative.  It  sums  these  six  values  and  sends  out 
a  single  identical  message  to  all  six  alpha  pos  machines,  which,  depending 
on  the  sign  of  the  sum  is  either  null,  or  an  increment  or  decrement  to  the 
current  o  position  of  each  leg.  The  alpha  balance  machine  samples  the 
leg  positions  at  a  relatively  high  rate.  Thus  if  one  leg  happens  to  move 
forward  for  some  reason,  all  legs  will  receive  a  series  of  messages  to  move 
backward  slightly. 

Next,  the  alpha  advance  ASFM  is  added  for  each  leg.  Whenever  it 
notices  that  the  leg  is  raised  (by  monitoring  the  output  of  the  beta  pos 
machine)  it  forces  the  leg  forward  by  suppressing  the  signal  coming  from 
the  global  alpha  balance  machine.  Thus,  if  a  leg  is  raised  for  some  reason 
it  reflexivelv  swings  forward,  and  all  ot  her  legs  ,wirig  backward  sliglitlv  to 
compensate  (notice  that  the  forward  'winding  leg  does  not  even  receive 
the  backward  message  due  to  the  suppression  of  that  signal ).  Now  a  tilth 
ASFM.  up  ley  trigger  is  added  for  each  leg  which  can  issue  a  command  to 
lift  a  leg  1 1 v  suppressing  the  commands  from  the  by  dmcn  machine.  It  has 
one  register  w  hich  monitors  the  current  i  position  of  the  leg.  When  i  is 
down,  and  a  trigger  message  is  received  in  a  second  register,  it  ensures 
that  the  contents  of  an  initially  constant  third  register,  are  sent  to  the 
In  tn  pos  machine  to  lift  the  leg. 

With  this  combination  of  local  leg  specific  machines  and  a  single  ma¬ 
chine  Irving  to  globally  coordinate  the  sum  of  the  o  position  of  all  legs, 
the  robot  can  very  nearly  walk.  If  an  up  leg  trigger  machine  receives  a 
trigger  message  it  lifts  its  associated  leg.  which  triggers  a  reflex  to  swing 
it  foward.  and  then  the  appropriate  leg  down  machine  will  pull  the  leg 
down.  At  the  same  time  all  the  other  legs  still  on  the  ground  (those  not 
busy  moving  forward)  will  swing  backwards,  moving  the  robot,  fowards. 

The  final  piece  of  the  puzzle  is  to  add  a  single  AFSM  which  sequences 
walking  by  sending  trigger  messages  in  some  appropriate  pattern  to  each 
of  the  six  up  leg  trigger  machines.  We  have  used  two  versions  of  this 
machine,  both  of  which  complete  a  gait  cycle  once  every  2.4  seconds. 
One  machine  produces  the  well  known  alternating  tripod  [5],  by  sending 
simultaneous  lift  triggers  to  triples  of  legs  every  1.2  seconds.  The  other 
produces  the  standard  back  to  front  ripple  gait  by  sending  a  trigger  mes¬ 
sage  to  a  different  leg  every  0.4  seconds.  Other  gaits  are  possible  by 
simple  substitution  of  this  machine.  The  machine  walks  with  this  net¬ 
work,  hut  is  insensitive  to  the  terrain  over  which  it  is  walking  and  tends 
to  roll  and  pitch  excessively  as  it  walks  over  obstacles.  The  complete 
network  fur  this  simple  tvpe  of  walking  is  >hown  in  figure  1. 
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3  Force  balancing.  A  simple  minded  wav  to  compenstate  for  rough  ter¬ 
rain  is  to  monitor  the  force  on  each  ieg  as  it  is  placed  on  the  ground  and 
hack  off  if  it  rises  hevond  some  threshold.  The  rationale  is  that  if  a  leg  is 
being  placed  down  on  an  obstacle  it  will  have  to  roll  (or  pitch)  the  body 
of  the  robot  in  order  for  the  leg  J  angle  to  reach  its  preset  value,  increas¬ 
ing  the  load  on  the  motor,  lor  each  ieg  a  beta  force  machine  is  added 
which  monitors  the  J  motor  forces,  discarding  high  readings  coming  from 
servo  errors  during  free  space  swinging,  and  a  beta  balance  machine  which 
sends  out  lift  up  messages  whenever  the  force  is  too  high.  It  includes  a 
small  deadband  where  it  sends  out  zero  move  messages  which  trickle  down 
through  a  defaulting  switch  on  the  up  by  trigger  to  eventually  suppress 
the  leg  down  reflex.  This  is  a  form  of  .ctive  compliance  which  has  a  num¬ 
ber  of  known  problems  on  walking  machines  [5].  On  a  standard  obstacle 
course  (a  single  a  centimeter  high  obstacle  on  a  plane)  this  new  machine 
significantly  reduced  the  standard  deviation,  over  a  12  second  period,  of 
the  readings  from  onboard  4  bit  pitch  and  roll  inclinometers.  Each  in¬ 
clinometer  had  a  la  degree  range.  The  standard  deviation  of  the  pitch 
inclinometer  fell  from  3.592  to  2.325.  The  standard  deviation  of  the  roll 
inclinometer  fell  from  0.521  to  0.451.  See  figure  5  for  details. 

■i  Leg  lifting.  There  is  a  tradeoff  between  how  high  each  leg  is  lifted 
and  overall  walking  speed.  But  low  leg  lifts  limit  the  height  of  obstacles 
which  can  be  easily  scaled.  An  eighth  AFSM  for  each  leg  compensates  for 
this  bv  measuring  the  force  on  the  forward  swing  (a)  motor  as  it  swings 
forward  and  writing  the  height  register  in  the  up  ley  trigger  at  a  higher 
value  setting  up  for  a  higher  lift  of  the  leg  on  the  next  step  cycle  of  that 
leg.  The  up  leg  trigger  resets  this  value  after  the  next  step. 

5  Whiskers.  In  order  to  anticipate  obstacles  better,  rather  than  waiting 
until  the  front  legs  are  rammed  against  them,  each  of  two  whiskers  is 
monitored  by  a  feeler  machine  and  the  lift  of  the  the  left  and  right  front 
legs  is  appropriately  upped  for  the  next  step  cycle. 

6  Pitch  stabilization.  The  simple  force  balancing  strategy  above  is  by  no 
means  perfect.  In  particular  in  high  pitch  situations  the  rear  or  front  legs 
(depending  on  the  direction  of  pitch)  are  heavily  loaded  and  so  tend  to  be 
lifted  slightly  causing  the  robot  to  sag  and  increase  the  pitch  even  more. 
Therefore  one  forward  pitch  and  one  backward  pitch  AFSM  are  added  to 
monitor  high  pitch  conditions  on  the  pitch  inclinometer  and  to  inhibit 
the  local  beta  balance  machine  output  in  the  appropriate  circumstances. 
The  pitch  standard  deviation  over  the  12  second  test  reduces  to  1.921 
with  this  improvement  while  the  roll  standard  deviation  stays  around 
the  same  at  0.  154.  Again  see  figure  5. 
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yy  y'VAv _ rVvw  w\^->A,.  beta  balance  std-2.325 

w v'  ' 'A - jA//\A‘N-  betabal  inhibit  std-1.921 


Figure  5.  The  robot  was  set  walking  across  a  plane  wit  ha  a  single  5cm  high  obstacle, 
file  graphs  above  record  the  measured  pitch  during  three  trials  each  for  a  duration 
of  12  seconds.  The  middle  of  the  range  of  each  graph  corresponds  to  a  level  body. 


I  he  upper  trace  corresponds  to  simple  walking  with  no  force  feedback.  The  middle 
trace  corresponds  to  walking  with  the  hi  In  huUmet  machine  included.  The  lower 
trace  corresponds  to  the  walking  machine  with  hi  In  bnlnnn  inhibited  in  high  pitch 
situations.  I  he  standard  deviations  for  the  three  trials  are  displayed  at  the  right. 


7  Prowling.  Two  additional  AFSMs  can  be  added  so  that  the  robot  only 
bothers  to  walk  when  there  is  something  moving  nearby.  The  IR  sensors 
machine  monitors  an  array  of  six  forward  looking  pyro  electric  infrared 
sensors  and  sends  an  activity  message  to  the  prowl  machine  when  it  de¬ 
tects  motion.  The  prowl  machine  usually  inhibits  the  leg  lifting  trigger 
messages  from  the  walk  machine  excent  for  a  little  while  after  infrared 
activity  is  noticed.  Thus  the  robot  sits  still  until  a  person,  say,  walks  by. 
and  then  it  moves  forward  a  little. 


8  Steered  prowling.  The  single  fitter  AFSM  takes  note  of  the  predomi¬ 
nant  direction,  if  any.  of  the  infrared  activity  and  writes  into  a  register  in 
each  alpha  pos  machine  for  legs  on  that  side  of  the  robot,  specifying  the 
rear  swinging  stop  position  of  the  leg.  This  gets  reset  on  every  stepping 
cvrle  of  the  leg,  so  the  steer  machine  must  constantly  refresh  it  in  order 
to  reduce  the  leg's  backswing  and  force  the  robot  to  turn  in  the  direction 
of  the  activity.  With  this  single  additional  machine  the  robot  is  able  to 


follow  moving  object'-  such  .i-.  a  -low  walking  person. 
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Conclusion 


This  exercise  in  synthetic  neuro-ethology  has  successfully  demonstrated  a 
number  of  things,  at  least  in  the  robot  domain.  All  these  demonstrations 
depend  on  the  manner  in  which  the  networks  were  built  incrementally  from 
augmented  finite  state  machines. 

•  Robusi  walking  behaviors  can  be  produced  by  a  distributed  system  with 
very  limited  central  coordination.  In  particular  much  of  the  sensory- 
motor  integration  which  goes  on  can  happen  within  local  asynchronous 
units.  This  has  relevance,  in  the  form  of  an  existence  proof,  to  the  debate 
on  the  central  versus  peripheral  control  of  motion  [7]  and  in  particular 
in  the  domain  of  insect  walking  [3]. 

•  Higher  level  behaviors  (such  as  following  people)  can  be  integrated  into  a 
system  which  controls  lower  level  behaviors,  such  as  leg  lifting  and  force 
balancing,  in  a  completely  seamless  way.  There  is  no  need  to  postulate 
qualitatively  different  sorts  of  structures  for  different  levels  of  behaviors 
and  no  need  to  postulate  unique  forms  of  network  interconnect  to  inte¬ 
grate  higher  level  behaviors. 

•  Coherent  macro  behaviors  can  arise  from  many  independent  micro  behav¬ 
iors.  For  instance,  the  robot  following  people  works  even  though  most  of 
the  effort  is  being  done  by  independent  circuits  driving  legs,  and  these  cir¬ 
cuits  are  getting  only  very  indirect  pieces  of  information  from  the  higher 
levels,  and  none  of  this  conununication  refers  at  all  to  the  task  in  hand 
(or  foot). 

•  There  is  no  need  to  postulate  a  central  repository  for  sensor  fusion  to  feed 
into.  Conflict  resolution  tends  to  happen  more  at  the  motor  command 
level,  rather  than  the  sensor  or  perception  level. 

As  a  last  observation,  there  are  a  few  straight-forward  engineering  improve¬ 
ments  which  could  be  made  to  the  current  robot  which  should  improve  its 
performance  markedly.  The  first  is  to  add  true  position  sensors  to  each  motor 
that  are  accessible  to  the  subsumption  architecture.  Currently  it  relies  on 
a  knowledge  of  mo«f  recently  conunanded  positions  of  motors  where  (espe¬ 
cially  on  rough  terrain)  this  may  not  correspond  well  to  the  actual  positions 
of  motors.  Secondly,  true  force  sensing,  using  strain  guages.  would  give  much 
more  reliable  force  readings  than  the  rather  indirect  met  hid  we  use  at  the 
moment  which  is  to  coarsely  time  the  switching  time  of  the  motor  current  as 
determined  by  a  build  in  analog  position  ^ervo  svstem  mi  each  motor. 
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